United States Patent [i9] 

Prasad et al. 



US005675511A 
[11] Patent Number: 
[45] Date of Patent: 



5,675^11 
Oct. 7, 1997 



[54] APPARATUS AND METHOD FOR EVENT 
TAGGING FOR MULTIPLE AUDIO, VTOEO, 
AND DATA STREAMS 

[75] Inventors: Rama R. Prasad; Ram V. Chary, both 
of Beavcrton, Qrcg. 

[73] Assignee: Intd Corporation, Santa Clara, Calif. 

pi] AppL No.: 576^19 
[22] Filed: Dec 21, 1995 

[51] Int CL* H04Q 3/54 

[52] U.S.CL 364^14 A; 348^ 

[58] FIdd of Seardi 364/514 R; 380/19; 

348/7, 467; 340/825.14; 370/60, 61, 47, 

99 

[56] References Cited 

U.S. PATENT DOCUMENTS 
4,688^47 8/1987 Davidov 380/19 



4,780 J58 10^1988 Linetal. ..... 

5,517^ 5/1996 Hoogenboom et al. 



348/7 

,.. 348/467 



Primary Examiner^-Elhs B. Ramirez 

Assistant Examinef^-Thonm Peeso 

AttOTTiey^ Agent, or F//t»— William H. Mmray; N. Stephan 

Kinsella 



[57] 



ABSTRACT 



A method, apparatus^ and storage medium for event tagging 
of at least one audio, video, or data (AVD) stream, each AVD 
stream comprising a plurality of ^D^ stream chunks. 
According to a preferred embodiment, the at least one AVD 
stream is stored. An event tag is stored in response to a signal 
indicating the b^inning of an event, the event tag having an 
event stream information data (ESID) structure comprising 
an ESID entry for each AVD stream of the at least one AVD 
stream, each ESID entry com{Hisiog an offset for locating a 
corresponding AVD stream chunk of each active AVD 
stream of the at least one AVD stream. 

50 Claims, 6 Drawing Sheets 
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APPARATUS AND METHOD FOR EVENT for playback. There is, therefore, a need for methods and 

TAGGING FOR MULTIPLE AUDIO, VIDEO, apparatuses that provide for event tagging for synchronized 

AND DATA STREAMS storing and retrieval of niultiplc A/V/D streams. 

^„ ^ ^^^^^^^ Addidonally, various A/V/D streams may be paused and 

BACKGROUND OF THE INVENTION 5 resumed independently of one another during a data con- 

1. Fidd of the Invention ference. There is, therefore, a need for me&ods and appa- 
The present invention relates to data processing systems, ratuscs for creation and management of multiple discontinu- 

and, in particular, to ^aratuscs and methods for providing f segments of different types of data (i.e. AJVfD) streams 

for event tagging for multiple audio, video, and data streams. i° * sin^t file. 

2. Description of the Related Art SUMMARY 
In computer systems, there is a need for storage and 

retrieval of various types of data streams. These data streams There is provided herein a mediod, apparsSOts. and storage 

can be audio data streams, video data streams, or other types medium for event tagging of at least one audio, video, or 

of (non-video, non-audio) data streams. There is c^n a need data (AVD) stream, each AVD stream comprising a plurality 

to store (and retrieve) miiltiple streams of such data, and to of AVD stream chunks. According to a preferred embodi- 

maintain the temporal synchronization of such data so that, ment of the invention, the at least one AVD stream is stored 

upon retrieval or playback, streams that occurred simulta- An event tag is stored in response to a signal indicating the 

neously ™intfli'n the correct time synchronization. Audio, beginning of an event, the event tag having an event stream 

video, and data streams sometimes referred to collectively 20 information data (ESID) structure comprising an ESID entry 

herein as ** A/V/D streams,** and storage and retrieval of for each AVD stream of the at least one AVD stream, each 

A/V/D streams is (also referred to as record and playback, ESID entry coursing an offset for locating a conespond- 

or write and read, of such data. In oon^uter systems such as ing ^D stream chunk of each active AVD stream of the at 

data conferencing (or 'teleconferencing**) systems, a plural- least one AVD streanL 

ity of users are provided with the ability to have an elcc- 25 

tronic on-line meeting even if the users are not physically in BRIEF DESCRIFnON OF THE DRAWINGS 

^e same room or building. Using application Programs for ^ advantages of the 

this purpose, modern o™ have enabled present invention wiU become more fuUy apparent from the 

the ability to have a "medtmg wherein aU users partiapate ^^^^ description, appended claims, aSdaccompanying 

in the mectmg through then- mdividual computer systems 3^ ^^wings in wh^: 

and share data, graphics, text, and other types of informa- „J!^^ ^ V. . * . ^ . 

tion.Usersmay oaimminicatewilhoncanothcr,sharingdata .^^'i^^^j "^^"^^"^ system m accordance 

in the form of graphical images, text, or other "annotations** * embodiment of the present mvention; 

and other infonnation represented on the computer system FIG. 2 shows a multi-stream temporal indexed file format 

display. This is analogous to a meeting where participants in 33 iA accordance with a preferred emtx>diment of the present 

a facc-to-fece meeting may display information to one invention; 

another on a whiteboard and other participants may add FIG. 3 depicts an illustrative data conference with a 

annotations, delete, or otherwise modify the board. In some plurality of data streams in accordance with a preferred 

usages, video data may also be shared among a plurality of embodiment of ttie present invention; 

connected users during such tdeconferenoes, or video con- ^ piG. 4 shows a multi-stream event tag file format in 

fcrences. For such an electronic conferencing system, it is acconiancc with a preferred embodiment of the present 

desirable for individual users to l>e able to selectively record invention* 

(and play back) any or dl of the individual AT/ZD streams ^'^^^ ^ ^ 

that constitule the data confaencc. dance with a prefiaredenibodinient of the present invention; 

Somecuirent methods such « those using the « p,G < ^^^^3 , commonieccri & play ar«4dtectuie, in 

^^ft^'^^^- ^ im^l. ^'^^ accordance with a prcfeircd cmbodiiLt of the present 

allow the recording and mdcxing of A/V/D streams. Current invention* and 

techniques utilizing such interleave formats, however, do * 

not provide an efficient means of locating (seeking) given exenq)lary receded file with multiple 

time-points within the AAr/Dreccrding,e5peciaUy when the » streams, in accordance witfi a prefened embodiment of the 

various streams are not all simultaneously recorded (e.g., present mvention. 

Z1^u^J^Si^'^i^l^'^<^ DESCRIPnON OFTHEreEFERRED 

continuously reooiaea). rurnier, once a paxticuiar location m EMBODIMENT 

the file has been seeked which correspcmds to a particular 

time-point, such cunent methods do not provide a means for 55 System Hardware 

selectively reading one or more cf the streams while main- Rcf ctring now to FIG. 1 , them is shown data conferencing 
taining synchronization between those streams. Thus, prior system IM, in accordance with a pfefeired embodiment of 
art indexing and synchronization methods suffer fipom sev- the present inveiition. Data conferencing system 100 com- 
eral disadvantages. There is, therefore, a need for methods prises a plurality cf nodes or personal computers (*PC**) 110, 
and apparatuses that provide fcr temporal indexing for so 120, 130. Personal computer or node HO con^ses a 
synchronized storing and retrieval oi multipie audio, video, processor 116, menuKy 117, video camera 111, micrcphone 
and data streams. 112, mouse 113, speakers 114, and monitor 115. Nodes 110, 
There is also a need to be able to retrieve and playback 120, 130 and other nodes <3i the data conference are inter- 
certain defined events that occur during a data conference. connected via medium 101. Medium 101 may be, for 
Current methods, such as those utilizing the AVI file format 65 exaxoplt^ a communication channel such as an Integrated 
standard, do not provide a means for tagging an event within Services Digital Network (*1SDN**)- As wiU be understood, 
a given A/V/D recording which may be randomly accessed various nodes of a conferencing system may be connected 
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through a variety of conununication media^ including local efiEtdcnt seeking to any specific point in time within a given 

area networks ("LANs"), plain-old telephone lines AfWID recording (ic. file), even when the various streams 

("POTS"), sonaetimcs referred to as public switched tele- are not all simultaneously recorded (eg., some streams arc 

phone networics ("PSTN*'), and the like. In a multi-point paused and resumed whUe others are still continuously 

'^^*^i^^^?T^r'«^^^^P^'^!I?^^M*?^^^^ ^ recoided).AswiUbcappreciated,aparticularAAr/Drecord- 

I'^^.u^'' ^ ""^^ ^ undffstood i comprises any number of au£o, video, and/cr data 

by those skUled in the art to transmit data to and from strWwhidi i^ed not have identk:al start times, end 

'^'^^IZ^t^'^t ^^^f^^ 'y'^ times, or lengths. TTie present invention further provikes a 

Nodes and/or MCUs may be mtcxconnerted via an ISDN ,1^^^ retrieving, in a fuUy syntonized 

Imkorthroughalocalareaw^^^ lO mamicr and at any given point^ time, any one or more 

various other communi^tions medi^ ITiose skiDed in the ^^J^ ^ ^^^^ ^^^^ 

^ r.'LT^L^o^^^^^ H coofarencmg systemmay, m aprefened embodiment, the p^3vention provides 

mgenaal, be com^ect^ f° the communications , „^ ^ temporS-index stream in a file, 

me<hum such as a LAN or through an MCU, and that a ^ ^ two AA^m^eLs are being recorded, 

con^crencmg systemmay con^mse other nodes or elements TOsaUows random access, or seek, to any time Jjintwiti^ 

su^ as routffs or servm, recording. Additionally, Ae present invention 

ftocessor 116 is preferably a general-purpose program- allows immediate recovery of one or more streams that was 

mable processor, such as an Intel® Pentium™ processor. recorded at that timi^JOint, while maintaining synchroniza- 

-niose skilled in the art will also appreciate that processors tion between the retrieved streams. Referring now to HG. 2, 

of nodes of conferencing system 100 may also be a special- ^ there is shown a preferred multi-stream tcimwral indexed 

purpose video processor such as the Intel® 82750PB. As ("MSTT) file format 200 in accordance wiTa prcfcncd 

^ be appreciated, the vanous peripherals and components embodiment of the present invention. As illustrated in HG. 

of a node sudh as those of node 110 may vary from those of 2, during recording of at least two AN ID streams by a user 

^""^ """^J^ and node m inay be config- of a data conferencing system. MSn file format 200 com- 

urcd identicaUy to or differently than node 110, as will also 25 prises a tcmporal-indot stream (^-) 201, having a TB 

be understood. It will further be understood that a node may stream header 205 and a pluraHty of TK chunks 209. Each 

be unplcmented on any suitaWe coix^xitcr system in addition XK chunk 209 represents a time-point in the recording. TIS 

to personal computer systems. chunks 209 are added at regular Intervals so that any 

As explained, each node 110, 120, 130 of a daU confer- particular moment during tiic conference may be accessed, 

encmg system is used by a user (or "agent") of the confer- 30 Each TB chunk 209 is of fixed size and is also relatively 

cnce. In a preferred conferencing system, a distributed small so as to allow for random accessing of particular TB 

^>pioach is used wherein each of the nodes maintains local chunks (corresponding to particular time-points in the 

copies of the c«if ercncing structore (sometimes referred to conference), 

as a "meeting"), which are to be consistent witii one another. MSTI file fcmat 200 also comprises a "helper stream" 
Using appUcation programs such as video conferencing 35 referred to herein as the stream infcmation dam ("SID") 
applications, aU users paitidpale in the meeting through stream 210, which conges an SID stream header 215 and 
tiicir individual PCS and share data, gr^hics, text, and other plurality of SID chunks 219 having a variable size. Each 
types of inf onnation. Users may communicate witii one (small, fixed-size) TB chunk 209 contains a pointer which 
another, sharing data in the fom of graphical images, text, points to a corresponding SID chunk 219. In a preferred 
or other "annotations" and other infonnation represented on 40 embodiment, each TB chunk 209 comprises a double-word 
a diq)lay such as one displayed on monitor 115 of PC 110. (ic, 32-bit) TB chunk time stamp field 209a and a double- 
Thus, as will be understood, in a typical data conferencing word (32-bit) TB chunk offset 209* into tiie SID stream 
session there are severalAA^yD streams of various types. At 210. As will be appreciated by those skilled in the art, 
any given time during the conference, tiicre are a particular timc-stanq) field 209a is tfic millisecond offset at which tfic 
number of such streams in existence. Each PJWiD stream 43 particular TB chunk 209 was generated. Thus, the time 
may have a start time, end time, and length independent of < stanip of time stamp field 209a of a particular TB diunk 209 
other A/WfD streams Uiat constitute the data conference. is the offset from the time that the TB stream 209 was 
Hius, some A/V/D streams overlap tenqxvally, some have created. - 

the same start and end times, and some do not ovcii^ The TB stream 201 is typically created at the same time 

temporally at all (Le. one ends before tiie otiicr begins). As so the A/V/D recording of the conference is begun. The crc- 

an exaiiq)le, users 1, 2, an 3 may begin and end a data ation time of the TB stream 201 itself is stored in the TB 

conference substantially simultaneously. The data confer- stream header 205, as illustiated by creation time field 205a. 

cnce may last for, say 30 minutes, from minute 0 to minute TB chunk offset 2$9b acts as the pointer described above. 

30, User 4 may enter at minute 1 and exit the conference at Thus, offset 209* provides forthe offsrt into die oorrespond- 

minute 10, while user 5 may enter the conference at minute 35 ing SID chunk 219 of tiie SID stream 210, as shown by 

20 and exit at minute 25. As will be understood, each user arrow 220 in FIG. 2. 

may have associated at least an audio and a video stream. Each SID chunk 219 comprises an array of SID entries 

Other streams may also be part of the conference, such as an 235, with one SID entry 235 for each stream that is being 

audio and video stream from a diird-party supplier, such as reamied by tiie user at tiic time of the event In a preferred 

a news channel, which is viewed or accessible by the «o embodiment, each SID entry 235 is 12 bytes long, and 

participants erf the conference. contains tiiree 32-bit fields 235a, 235fc, 235c. As wiU be 

Temporal Indexing appreciated, A/V/D streams are stored in memory as a 

In a preferred embodiment, there is provided a method stream of data faroJKn into smflllM- units, in accordance with 

and apparatus that provide for tcn^wal indexing for syn- tiic type of dau stream. These discrete units of a stream, or 

dironizcd storing and retrieval of multiple audio, video, and 65 chunks, may consist of, for example, video frames or 

data streams. As described in further detail hereinbelow, tiie pictures for video streams, or audio samples of a given 

present temporal indexing invention provides a means for lengtti (eg., one-fifth or one-tenth of a second) for audio 
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Streams. Each chunk of a stream thus corresponds to a time-point corresponds to a TIS chunk 2W and also to an 

uniquctime-pointduiingthcconfcrenceatwhichthatpartof SID chunk 219. The corresponding SID chunk 219 com- 

the stream occurred. Offset field 235a is, therefore, a pointer prises information about the state of the conference at the 

to the oonesponding chunk for the stream conesponding to corresponding time-point, such as the number and types of 

the SID entry 235, where the chunk corresponds to the 5 currently-existing PJSfD streams and their locations in 

time-point in the stream corresponding to the particular SID memory so that any or all of these A/V/D streams may be 

entry 235. As will be understood, each AA^/D stream of a selectively retrieved by a user desiring to play back all or a 

conference is associated with a unique identifier by appli- portion of the conference. In a preferred embodiment, each 

cation programs supporting such conferences. Stream ID user, or participant, in the data conference has the ability to 

field 235fc thus conqmscs information identifying the cor- lo selectively record any or all of die PJWID streams of the 

responding AA^/D stream. Stream status field 235c indicates conference. During such recording, or storage, of the 

the status of the associated stream, Le. whetho: the stream is selected AA^/D streams, US stream 201 and SID stream 210 

active or not (i.e. whether it is being recorded), or whether are also created and stored, as described hereinabove. Each 

the stream, if active, is currentty paused or not (Some selected AA^A) stream to be recorded in a given sequence of 

streams may temporarily be paused, ox muted, by the user 13 the conference is stored indcpendentiy, rather than mixed 

during recording for various reasons.) togedier, to allow for selective playback of any or all of die 

Each SID chunk 219 is thus of variable size because recorded AA^/D streams of the sequence, 
different numbers and types of A/V/D streams nuiy exist, As will be understood, the time stamp information st<xcd 
and/or be recorded, at given time-points of the data confer- in time stamp filed 209<i for each tinoe-point may be utilized 
ence. For exaix^le, at a first time-point in a data conference, 20 by die user's data oonfercnciiig system processor to syn- 
there may be three audio streams and one video stream, chronize all retrieved fields. For example, after the data 
\^1ule at a later point in time there may be a larger or smaller conference is ended and stored in a user*s memory, he may 
rumiber of such types of A/V/D streams. Referring now to seek to time and select retrieval and playback of only 
FIG. 3, there is shown an illustrative data conference 300 streams A^, V^, and Aj, thus omitting streams Aa and Ds- 
with a plurality of A/V/D streams in accordance with a 25 Thesefirstthreestreamsmay be retrieved firom memory and 
preferred embodiment of the present invention. As synchronized temporally utilizing the infonnation contained 
illustrated, at time To, the data conference comprises three in the corresponding TIS chunks 209 and SID chunks 219. 
users. Each participant in die conference is able to view the The present tenqxval indexing invention therefore pro- 
video stream V^ of uso^ 1, and the audio streams A^, A^, and vides a means for recording sequences of selected, multiple 
A3 of each of users 1, 2, and 3, respectively, may be heard 30 A/V/D streams with temporal indexing to aUow fos efficient 
thrxMigh speakers on each user's data conferencing systeno. seek and synchronous playt>8ck of selected A/V/D streams 
Additionally, a data stream Dj Of user 3 may also be shared of the recorded sequence. With this invention and corre- 
by each participant in die conference. Thus, the users may be sponding MSTI file format 200, a user using a browser/ 
sharing only the video stream originating from user 1, if die playback application can rapidly seek to any point in time 
local video of the other users is not relevant, but listening to 35 within an A/V/D reo(Mxling and have immediate synchro- 
the audio streams of all users so that their comments about nized access to any of the streams diat were recorded at that 
video stream Vi may be heard. User 3 may also be siq>plying point As will further be appreciated by those skilled in die 
a data stream D3, such as mouse pointer movements or the art, the present temporal indexing invention may be easQy 
like. Thus, at any time from time To to T2, such as time T^, implemented widiin the context of industry-standard file 
the data conference comprises five total A/V/D streams, 40 fonnats such as die Miaosoft® AVI file format 
namely one video stream, three audio streams, and one Further details conccniittg a preferred einbodirnent of the 
(non-video, non-audio) data stream. At time T,, user 4 may present invention are presented in die Record and Playback 
join the conference and exit at time T^, as illustrated Hius, Services Definition section and File Input^Ou^t Services 
at time T3, there is an additional audio stream, A4. Similariy, Deftniti<m section of this application, below, 
at time T4, an A/V/D source such as a news source having 45 Event Tagging 

streams A5, V5, and D5, may be added to the conference. As explained hereinabove, there is a need to be able to 
These streams may represent, for exanq)le, audio and video retrieve and playback certain defined events that occur 
data displaying information from a cable news chaimel, and during a data conference. In a preferred embodiment of the 
data stream D5 may re{^sent stock ticker information, present invention, there is provided a method and apparatus 
inftxmation about the content of odieravailat^e channels, or 50 for event tagging and fcr syndironized storing and retrieval 
tiie like. of multiple A/V/D streams. In a preferred file format in 
The time during the conference is divided into a phuali^ accordance with the present invention, a nuilti-sticam event 
of discrete, small time intervals sufiBcteiit to allow accurate tag (MSET) format is utilized. Refening now to FIG. 4, 
accessing of selected timei)oints in the conference. For there is shown an MSET file fonnat in accordance wldi a 
example, the time-points may conespond to each tenth cr 55 preferred embodiment of the present invention. MSET file 
half of a second. As will be appctdatal die smaller the format conqirises event tag 400, which is added during 
intervals represented by such time-points, the more such recording of an A/V/D conference or sequence coursing a 
time-points and associated processor time and memory sut»et of the streams of the conference in die maimer 
storage will be required. However, with smaller time described hereinabove. Event tag 400 comprises an event ID 
intervals, more precise seeking of specific ixxmients of die 60 field 401, time stamp field 402, and event stream informa- 
recorded data conference may be achieved. Conversely, a tion data ("ESID**) structure 410. ESID structure 410 coin- 
more coarse time-point (Le^, larger time-intervals) will prises number of entries field 412 and an array of stream 
require less processing and mem<My, but will not allow as index data ("SID") entries 415. Each stream index entry 415 
free cr precise seeking of the recorded streams of the comprises an offset 415a , stream ID 415^, and stream status 
conference. Additionally, widi coarser tiirie-points, synchro- 65 415c. 

nization of the streams desired to be played back will be Event ID field 401 is preferably a double-word (32-bit) 

more difficult to achieve. As will be understood, each such unsigned integer, with 0 being a reserved value. This ID can 
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be used by application programs to map a symbolic name to application, such as a data conferencing application, to tag 

the event As wiU be understood, for each event tagged key events dining the recording of conferencing sessions or 

during a recorded sequence of a data confaence, an event sequences having subsets of ttie streams thereof. SymboUc 

Ug 400 IS aeated, each having a unique ID 401. This may ^mes may be assigned to each event and mmped lo the 

be mapped to a symbolic name such as "John's j event tags for cadi event Thereafter, a tagged^^may be 

hT^,^* « '^T ^J^' '^" effldenfly nindomly accessed, ie. s^Any or all of the 

Rand and Ludwig von Mises debated here " and the like, to c*^^«^c I^^a^ L 7 * « • JTi V ^ 

describe the event being tagged. TOs tag is typlcaUy added '^^J^^ a ^ J'T^ then immediately be 

for each defined or sdJted^ent. In ^Ical^^a S ^^^"i ^ maktaming sjmctoniz^ 

is generated which indicates that a new event is b^giiLig ^1? all such streams, since the offset 415fl for each 

for which an event tagc 400 should be created and itw^i ^ timc^wint to be located 

where the event signal is generated by a user. Alternatively, . **** A/V/D stream. With this invention and cooespond- 

cvents may be generated automatically when certain types of MSET file f<nnat, a user using a tawsec/lplayback 

events occur, for example when the scene changes cr when application can rapidly seek to any tagged event within an 

certain computer recognizable images,5ounds, cr the like AA^/D recording and have immediate synchronized access 

occur. toany of the streams that were recorded at that point As will 

The time-stamp field 402 is preferably a double- word fuitfaer be appreciated by those skilled in the art, the present 

field that marks the absolute time of the beginning of the event tagging invention may be easily in^)lemented within 

event being tagged. As will be understood, in alternative the context of industry-standard file foimats such as the 

preferred embodiments relative timo-stamps may be utilized Microsoft® AVI file format 

as wdL As will be appreciated, a sequence of events tagged 20 The present event tagging invention therefore provides a 

may be stared and tracked in an event track, so that selected means for event tagging and for synchronized storing and 

events of a plurality of tagged events may be seeked when retrieval of multiple ATV/D streams, 

desired by a user playing back a recording of the conference Further details concerning a preferred embodiment of the 

present invention arc presented in the Record and Flyback 

The ESID structure 410 comjMises an array of SID entries 25 Services Definition section and File Input/Output Services 

415, with one SID entry for each stream that is being Definition section of this apfdication, below, 

recorded by the user at the time of tfie event Hius, if seven A/V/D Stream Segment Management 

streams exist in the conference at a given selected event at As explained hereinabove, multiple A/V/D streams may 

whidi time during the recording the user creates a oorre- be recorded in a single file using a variety of suitable file 

spending event tag 400, the user may have selected, few 30 format techniques. When certain of the recorded streams arc 

example, only five of these seven streams to be recorded, as paused and resumed during recording, for example during a 

described in detail hereinabove with reference to FIG. 3. In data conference, discontinuous streams are created and need 

this case, there will be five SID entries 415, one for eadi oi to be managed for record and playback. A continuously 

the recorded streams, and number of entries field 412 wiU recorded portion of a stream (c.g., between pauses) is 

store the value five to indicate the number of such streams. 35 referred to herein as an AA^/D stream segment or segment 

In a preferred embodiment, each SID entry 415 is 12 bytes There is a need for accessing each segment independently, 

long, and contains three 32-bit fields 41Sfl, 415fr, 415c. In an or to seek to a specified point (referenced either by time Or 

alternative preferred embodiment there is an SID entry 415 by event tagging, as explained hereinabove) in any or all of 

for each stream diat exists in the conference, whether or not the streams in such a file. There is, therefore, a need for 

it is being recorded, but the SID entry 415 for unrecorded 40 methods and apparatuses for creation and management of 

streams indicate, as explained below, that the stream is multq>le discontinuous segments of different types of data 

inactive. (Lc. not being recorded at the moment the event is (Le. A/VfD) streams in a single file. This is aoconipUshed in 

tagged). the present invention by use of a special "directory" stream 

As WiU be appreciated, A/V/D streams are stored in that specifies a format to record information about every 

memory as a stream of data broken into smaller units, in 45 stream in a file. This directory stream can then be used by 

accordance with the type of data stream. These discrete units applications that play back such a file to display directory 

d a stream, or chunks, may consist of, for example, video infOTnation, and to locate specif evcn^ or time-points in one 

frames or picnires for video streams, or audio samples of a or more streams, thus allowing seek operations at the stream 

given length (e.g., one-fifth or one-tenth of a second) for or file level. 

audio streams. Eadi chunk of a stream thus corcsponds to 50 Referring now to FIG. 5, there is shown a stream segment 

a unique time-point during the conference at which that part directory table ("SSDT^ 500 in accordance with a prefaxcd 

of ttie stream occurred. OflFsct field 415a is, therefore, a embodiment of the present invenUon. SSDT 500 ccHi^rises 

pomter to die corresponding chunk for the stream corre- an array or table of stream segment directory ("SSD**) 

spending to the SID entry 415, where the chunk corresponds entries 510, one SSD entry 510 for each continuous segment 

to the time-point in the stream at the moment of the event 55 in the file. SSDT 500 also comprises a table size entry 501 

tag. As WiU be understood, each AA^/D stream of a confer- which indicates the total number of SSD 510 entries in 

encc is associated wi± a unique identifier by appUcation SSDT 500. As wUl be appreciated, table aze 501 is useful 

programs supporting such conferences. Stream ID field 415fc in aUowing an appUcation program to determine the amount 

thus ooBoprises information identifying the corresponding the memory space SSDT 500 needs, when used during 

A/V/D stream Stream status field 415c indicates the status «o processing of a recorded file. 

of the associated stream, i.e. whether the stream is active or Each SSD 510 comprises six fields, 5100^10/, which 

K)t(i.e.whetheritisbeingrecQrded),orwhetherthestream, identify and characterize each continuous AA^/D stream 

if active, is currenUy paused or not. (Some streams may segment of the file. As wiU be understood, each /JV/D 

temporarily be paused, or muted, by the user during record- stream of a conference is associated with a unique identifier 

ing for various reasons.) 65 by q)pUcation programs supporting such conferences. 

As wiU be understood by those skUled in the art, the event Stream identifier 510te , preferably a 32-bit field, thus 

tagging of the present invention may be used by an con9>rises information identifying the corresponding AAT/D 
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Stream from which the cocresponding segment is doived. RFFile 603 implements the high level file I/O API which 

Segment number 510^, I^eferably a 32-bit field,.is a number is discussed below. Various "service providers,** hooked 

from 1 through where N is the total number of segmente under RFFile 603, in4>lenient the actual support for these 

in the file. Thus, segment number 510^ is a unique segment ugh level APIs in tiie specified low level file format In the 

number for identifying the segment associated with tiie 5 present exan^)le as illustrated in HO. 6, the service provid- 

particular SSP 510. Stream state qualifier 510c is preferably ^rs shown arcTAPFilc Service ftovider ("SFO 612, AVIFilc 

a 32-bit field, and is used to indicate the status of the ^13^ PCSHle SP 606. When AVDAM 601 requests 

associated stream, i.e. whetoer tiie strcan is active or not creation/opening of a file through the record & play 

(Lc. whether it is being recorded), or whether the stream, if ^ ^ ^ ^3 ^ 

active, is anrently paused or not (Some streams, as skilkd in the art Based on the 

^plamed above, may tcrajxiranly be paused, or muted by file faimit, RPFile 603 loads the appropriate SP 

flic user during recording for vanous reasons.) Start tune resucawu mo "'^ ^y rTr; 

field510iandlndtimefield510*areeachpref«ably32-bit f^f «™« ^ Rations n<^<kd m^±c 

fields, and are used to denote the start and end times of tfie help of this service provider. The undfflymg file TO is 

relevant segment Offset field 510f is an offset pointer, designed to support an arbitrary number of streams. Each of 

preferably 32-hits in size, that locates the beginning of the 15 these streams is unique, wifli its own "streamlD," which 

coiresponding segment of tiiis stream in the file. corresponds to flie 'tourCC" code found in flie AVI file 

As will be understood, SSDT 500 may be stored as a format The record & play services in aocordance with flie 

separate stream wifli its own chunk identifier ('"ChunklD") present invention proposed adds support for the following 

in file formats such as the AVI format referenced herein- features, in addition to the basic record and playback capa- 

above. In other suitable file f(Hmats that support multiple 20 bilities: Support for **file-5cope** information like specific 

streams, wheflier interieaved or otherwise, an identifying annotationsAext data; support for "segment** infonnation for 

mechanism similar to ChunkID may be used to store SSDT each stream, allowing discontinuous stream recording and 

500. Therefore, flie present invention allows multimedia, or playback; support for 'Hmel^ack** (*TT*), allowing sup- 

A/V/D, plications to flexibly and efiSdently support port for efficient search and seek mechanisms across all 

record and playback of a variety of A/W/D streams, and 25 streams (file-level) or per stream; support for recording and 

allowing the creation of multiple segments within each using **Evenr markers, with appropriate stream information 

stream that can result from pause/resume operations. The appended; and gllobal file scope information regarding the 

present invention thus serves as a general mechanism for number of streams, specifics on each stream present, and the 

creation and management of stream level information in a like. 

file that is used for recording mult^le, discontinuous 30 Refecring now to FIG. 7, there is shown an exeinplary 

streams. Using SSDT 500, there is gained the c^faility to recorded A/V/D file 700 with multiple streams, in accor- 

access randomly, as opposed to sequentially, any stream dance with a preferred embodiment of the present invention, 

segment, thus making efficient seek operations possible. As illustrated, some of the streams are continuous while 

Also, the present invention thereby allows creation and use some have one or more discontinuous segments. All the 

and/or retrieval of directory information of streams and their 35 information on different streams and their segmentation is 

segments in a file. maintained in a file-scope stream segment information track. 

Fuither details concerning a preferred embodiment of the The underlying file I/O system supports creation and reading 

present invention are presented in the Record and Playback of such file-scope global information. As illustrated in FIG. 

Services Definition section and File Input/Output Services 7, VI 710 and V2 714 are video streams, and VIGD 710a 

Definition section of this application, t>elow. 40 and V2GD 714a represent video stream global data. Al 711 

Record and Playback Services Definition and A2 716 are audio streams, and Dl 713 is a data stream. 

The following provides a description of a preferred record TT 720 is a time track stream and TH 725 is its internal 

and idayback services definition in aoc<xdance with a pre- stream. Because flie underlying Hie I/O supports file-scope 

ferred embodiment of the present invention. The nomenda- global data and stream-scope global data read/write 

ture employed hereinbelow will be understood by those 4S c^bilities, it is easy to add any additional information like 

skilled in flie art Several types of tfiplications require record text annotations at the file scope and stream-specific infor- 

and playback support for recording A/VA> streams. Hie mation (e.g: KeyFrame markers in video streams), 

following describes a sharable set of senrices/modules fliat Below are described a prefoied API for record services 

all such projects may use and an associated common record and playback services, in accordance wifli the present inven- 

& play architecture 600 which is illustrated in FIG. 6. An 50 tion. 

associated high level file input/output (**l/0**) API is Reccrd Services 

described in tfie f blowing section. Described in fliis section jf^ record services described below arc divided into two 

is the record and playback services layer, and the related set categories, file scope and stream-scope operations, 

of fimctions needed by such rccad and play services to pile^sccpe operations 

su^ record and playbadc requirements. 53 intReccrdOpen(INLPCSTRlpPafli, IN DWORD dwFlags, 

T^foUowing «eamplesjm.«to^^^ apormat, OUT LP_FILE_HANDLE 

record & play architecture 600. Various apptications may be h>hFlleflandle) 

utilized with data conference systems. For example, an , ^ ^ . ^ ^ , ^ ^. 

AT/ZD answering machine CAVDANT) application may be "^^f"^^^ * tcrmmated pafli for flic file to 

used to record and play messages left by one user on another 60 created 

user's data conferencing personal node. This AVDAM is an dwFlags — specifies creation flags (APPEND/READ, 

example of an application that can use the record and play WRITE etc.), segokent/directory info creation etc. 

services, which in tum uses flie high level file I/O API uFormat— file fonnatto be used (.AVI, KS, .TAP, etc) 

described in the following section of this application. Refer- IphFileHandle— non-NULL if successful; NULL oth- 

ring now to FIG. 6, there is shown diagram 600 which 65 erwise 

illustrates the record and play architecture, ui accordance Return value: zero if successful; oflierwise, an error code 

with a preferred embodiment of the present invention. such as 
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ERRORJILE^CREAnON dwBufferSire— number of bytes to be written, contained 

ERRORJAD_j\RGS in IpData 

ERRORJFORMAT^UNSUPPORIIBD IpdwBytcsWritten— actual count of bytes written is 

int Rcc<ydClose(IN FIIJS_HANDLE hFilcHandlc) returned in feis location. 

Closes the file identified by hFileHandle. Also, 5 int RccordStreainPause(IN STREAMJIANDLE hStream) 

RPFileWriteFiieInfo( ) is invoked with the FIIXJNFO Pauses the recording of the specified stream; also, updates 

structure duly filled up, to write out general information on the segment/directory information accordingly. 

*isfile. int RecordStreamRcsume(IN STREAM_HANDLE 

return value-zero, if successful hStream) 

Return value: zero if successful; otherwise, an error code lo Resumes the recoiding of the specified stream; also, 

such as updates the segment/directory information accordinglty. 

ERROR_FILE_CXOSING Playback Services 

HRROR_JBAD_ARGS Playback sendees are divided into two general categories: 

int RecordFileEnable(IN FILEJIANDLE hFileHandle) file-scope operations and stream-scope operations. 

This function enables "streaming** of frames to all the File-scope Operations 

open streams specified in the file pointed to by hFileHandle. int PlayOpen(IN LPCSTR IpPath, IN DWORD dwFlags, IN 

This is usefiil especially with audio and video streams. UINT uFormat, OUT LP_J1LE_.HANDLE IphFileHandle) 

int RecordPause (IN FUJBuJiANDLE hFileHandle) lpPath--specifies a mJLL terminated path for the file to 

Pauses the recording of all open strean^; also, updates the be opened 
segment/directory information accordingly. » dwFlags-^pecifies file open flags (READ/READ 

mt RecordResume(IN FIUE_JiANDLE hFileHandle) WRITE etc) rex — 

Resumes Ae recording of all open stroims; also, updates uF6rmat-file fonnat to be used (.AVI, .PCS, .TAP, etc.) 

the segmcnt/dircctoiy information accordingly. , „ „ 

int RecordTimeTrackEnableON FILE_JIANDLE hFUe- IphHleHandlc^on-NUIX if successful; NULL other- 

Handle , IN DWORD dwlntcrval) 25 wise 

Creates/continues a llmcTrack (see FileI/0 API) with ^ ^^^^ * "^^^ 

pcr-stream information stored at dwintcrval periodicity. This ERROR_JTLE_J^0T_FOUND 

is done untU a Recor<rrimc'rrackDisable( ) or Recoidaosc( ERRORJAD_ARGS 

) is invoked, ERRORJORMAT^MBMArCH 
int RecordTimeTrackDisable(IN FILE_HANDLE ^ int Playaose(IN FILE_J1ANDLE hFileHandle) 

hFileHandle) Ooses the file identified by hFQeHandle 

Discontinues TimelVack return value-zero, if successful 

int RecQrdMarkEvent(IN FILE_JiANDLE hFileHandle, IN Return value: zero if successful; otherwise, an error code 

DWORD dwEventID) sudi as 

Creates/adds an EvcntTrack (see File 1/0 API) with ttic ERRORJILLCLOSING 

specified EvenOD, storing per-stream information for all ERRORJAD^JKRGS 

9pcn streams int PlayGclFileInfo(IN FILE_HANDLE hFileHandle, OUT 

Strewn opwations LP_JFILE_JNFO IpFilelnfo) 

int RecordSfreampeate(IN^ I^^ Returns the fil<ssV infcination on this file specified by 

^x>^^?J^Ji"i^/c*^ hFncHandle.lTieiii&tionisfiUedintothel^_JNFO 

LP.^STREA\LJNFO ^pStreamlnfo) IpFilelnfo. 

ll*StreamHandle-pointer to a StreamHandle IpFUelnfc^-pointcx to a buffer of type FILE JNFO 

uStreamiypc— specifies Video/Audio/Data stream type i^t PlayEnable(IN FILE_HANDLE hFileHandle) 

IpStreamlnfo-iKMnter to an ovcriay structure indicating This function enables "streaming" of flames &om all the 

stream-specific information streams specified in the open file pointed to by hFileHandle. 

This funcliwi sets the handle of the stream, if created This is useful especially with audio and video streams. 

successfiiUy, to a non-NULL value at IphStreamHandle. im PlayPausc(IN FILE_JL\NDLE hRleHandlc) 

Otherwise, a NULL value is set at this location. Pauses playing <rf aU open streams; also, imdates tiie 

int RecordStreamClose(IN STREAMJANDLE ^ segment/directory infonnation accordingly. 

hStreamHandle) int PlayResume(IN F1LE_JHANDLE hFileHandle) 

hStrcamHandlc— StreamHandle Resumes the piaying of all open streams ; also, updates the 

This function closes the specified stream. segment/directory infonnation accordingly. 

int RecordStTcamEnable(IN STREAM_HANDLE int PlayGctTimeTrackInfo(IN FILE_HANDLE 

hStreamHandle) j5 hFileHandle, OUT LP_TIME__TRACK 

hStreamHandle— handle to an open stream IpTinu^VackBuffcr, IN DWORD dwBufSize) 

Enables the recording of the specified stream; also. This function is to be used to obtain 'HmeTYack** infer- 

updates the segment^directory information accordingly. mation from the file. 

This function allows streaming of audio/video frames to IpTimcltackBuffe— a pointer to a buffw to read in the 

the appropriate stream in the file, if the stream handle ^ "Ilmctrack" information 

spedfies either an audio or a video stream. dwBufSize^-spcdfies the size, in bytes, of the buffer 

intRecordStreamWritedNmEAMJIANDLEhStieam, pointedtobTSr^ r me Duircr 

wrif^TZi^^i^^.^^y^'^^^®"^'^^'^^ Pl«y GetEventTrackInfo(IN FILE HANDLE 

WORDlpdwBytesWntten) hFileHandle, OUT LP_TIMe"1tRACK 
hStreamHandle-^andle to an open stream 65 IpEventmctcBuffer, IN DWORD dwBufSize) 

^Data— pointer to a buffer containing data to be written This function is to be used to obtain "IlmcTVack" infor- 

to the stream specified mation from the file. 
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IpTimeTrackBuffe — a pointer to a buffer to read in the a variety of purposes, including visual display of "slider 

*Tinietrack" information bars** for each segment in a stream. 

dwBufSize— specifies the size, in bytes, of the buffer hSbceamHandle^andle to an open stream 

pointed to by IpTuneTrackBuff^ ^egmcntlnfo — pointer to a SEG_JNFO structure, to 

int HayHIeSeekBvent(IN FHJE_JIANDLE hFileHandle, 5 segment info 

IN DWORD dwEventlD) int PlayStrcamSeck(IN STREAM_HANDLE 

This function results in a seek operation to be done on the hStreamHandle, IN DWORD dwOffSet) 

file. All Che streams are positioned at the offsets obtained This function results in a seek operation to be done on the 

from an Event information entry, specified by dwEventlD. specified stream. The stream is positioned at the offset 

int PlaymeSeeiaime(INFILE_HANDLE hFileHandle, IN specified by dwOffSet. 

DWORD dwTimcStanq)) int PlayStrcamSeekEvent(IN STREAMJANDLE 

This function results in a seek operation to be done on the hStreamHandle JN DWORD dwEventlD) 

file. All the streams are positioned at the offsets obtained This function results in a seek operation to be done on a 

£rom the TimeTrack entry, specified by dwUmeStamp. stream The stream specified by hStreamHandle is posi- 

Stream infonnation functions tioned at die offset obtained from an Event information 

The functions defined below assist in obtaining global entry, specified by dwEventlD. 

information on supported streams, without opening any int PlayStreamSeekTime(IN STREAM ^HANDLE 

specific streams. hStreamHandle, IN D WORD dwTimcStainp) 

int PlayGctFirstStrcamInfo(IN UINT uStreamTVpc, OUT This function results in a seek operation to be done on a 
LP_JSrREAM_JNFO IpStreamlnfo, IN BOOL bMorc) » stream. The stream specified by hStreamHandle is posi- 

Reads the Streamlnfo for the **first" instance of the tioned at the offset obtained ftom a TimeTrack information 

specified type of stream (audio, video or data); bMc^ entry, specified by dwllmeStamp. 

indicates if there are more streams of the qjedfied type in int PlaySeekNextSegment(IN STREAM_HANDLE 

^ file. hStreamHandle) 

int PlayGetNextStreamInfo(IN UINT uStreamTVpc, OUT " This function results in a seek operation to be done on a 

LP_STREAM_JNFO IpStreamlnfo, OUT BOOL hMore) stream The stream specified by hStreamHandle is posi- 

Reads the Streamlnfo for the "nexr instance of the tioned at the offset obtained from a Stream Segment infor- 

spedfied type of stream, relative to the stream specified in mation entry, identified by this stream's ID. 

previous PUyGetFirstStreamlnfo/PlayGetNe^tStreamlnfo; Stream Read functions 

bMore indicates if there are more streams of tiie specified PiayStreamRead(IN STREAM_HANDLE 

type in this file. hStreamHandle, OUT LPBYTE IpBuffer, IN DWORD 

Stream Open functions dwBufSize, OUT LPDWORD IpdwBytesRead) 

int PlayStreamOpen(IN DWORD dwStreamID, OUT hStieamHandle-liandle to ao open stream 

LPSTREAM_JIANDLE IphStream) , ^k-*. * * u « \1- • ^ . . ^ 

^^j^^<^n<^^.^sp^^r^^uc^, 3. dau to be read 

^ ^ ' ^ . . dwBufSize size of the buffer pointed to by IpData 

dwStreamID — specifies the stream to be opened ijTi.*nj ^. 

.^^^ IdwBytcsRead — actual count of bytes read is returned in 

IphStream— pomtcr to a STREAM JIANDLE; filled in this location. 
wi& the stream handle if nayppcnStteam is success- ^ i^t PlayStreamEnable(IN STREAM_HANDLE 

^ hStreamHandle) 

int PlayStreamClose(IN STREAM_HANDLE jhis function enables "streaming" of frames from the 

hStreamHandle) ope^ stream specified by hStreamHandle. This is usefiil 

hStreamHandle — StreamHandle especially with audio and video streams. 

This function closes the specified streanL 45 int PlayStreamPause(IN SrREAM_HANDLE hStream) 

Stream level seek functions Pauses the recording of the specified stream; also, updates 

int PlayGetSegmentInfo(IN STREAM.UANDLE the segment/directory information accordingly. 

hStreamHandle, OUT LP__SEG_JNFO IpSegmcntlnfo) int FlayStreamRcsume(IN STREAM__HANDLE hStrcam) 

This function is used to obtain "segment** information on Resun^s the recording of the specified stream; also, 

an open streanL The infotmatioo obtained may be used for updates the segment/directory information accordingly. 



Data Structures uaed 



stnict TtmelhckS 
{ 

DWORD dwTimeStamp; //(relative/m^)tiine at Ihb point, 

/y in miUisecoads ' 

DWORD dwOftemmelkaciklnfo; //offKt ma> TimetDsclc stiaun 
}'nM9_TRACK; 
struct StreunhickcxEatiy 
{ 

DWORD dwStreamID; //fbufOC 

DWORD dwStzesmStatus; //active, puisBd . . . 

DWORD dwOfict; /rtm^hr 
ISrREAMJNDEX-EtORY; 
struct IrmcTnfbChpnk 
{ 

DWORD dwNumlDdekEatries; //$»nc as # of stzesms open now 
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Data Structures used 



SrnCEAKCJNBEX-^ENTRV aStrBaiiiIiidcxEiilxies[dwNuizibxfexEiilrics]; 
IHME-THACXJNFO; 
struct EvenUnfo 
{ 

DWORD dwNumlDdetEntTies; // # of open streams 
SnrREAM_JNDE}C_fNTKr aStreamliicfexEzmies^dwNiiiiibidexEatries]; 

)EVENT_JNFO; 

struct EvenOWclcS 



{ 

DWORD dwEventXD; 

DWORD dwUmcStamp; 

EVENT_JNFG sSventlnfo; 
}EVENT_niAClLJNFO; 
struct StreamS^gDir 
{ 

DWORD dwStreamID; 

DWORD dwSegmeotKuaaber; 

DWORD dwStreamStste; 

DWORD dwSegStaxtMark; 

DWORD dwSegEndmatk; 

DWORD dwStreamOffSet; 
}STREAM»SBG_J>nUBNTRY; 
struct SircamScgDizlVibk 



//event ID - uniquB 
//time at this point 
//aptaxes all other streams* info 



//same as four CC 

//activet paused . , . 
//time, event 
//time, event 
//offset into stream 



{ 



DWORD dwNumDiiEatnes; // (» of streams) X (segments in 
// eachstxcazn) 



ST1tEAM-JSiEO_J)0L_ENTRY aStreamSegDtiEntry[dwNumDi£atrse8]; 
}STREAM_SBG__DnUTAHLE; 
struct Streamlnfol)^ 



{ 



DWORD dwStreamID; // fouzCX^ code, determined by uStreamType 
unjoD StreamSpeificInfo 
{ 



VIDEO_JNFO sVideoJafo; 
A1}DI0_JNF0 sAudioSofo; 
DADLJNFO sDatalofo; 
}STREAM_SPECIFlC_JNFO; 
ISTREA&IJNFO; 
struct Fiklufo'Ihg 
{ 

UINT uFormat; 
iU nNunibarOfStreams; 
BOOL bUmelkackPresait; 
DWORD dwNum'HmelhckEntries; 
BOOL bEvcntTrackPrescnl ; 

DWORD dwNumiEventinad^ntries; 
int nNumberOfVidBQStreams; 
int nKumfoerOfAudioStreams; 
int oNumberOfDataStreams; 
VIDEO_JNF0 aVideoInfoCnNumbefOfVideoStieania]; 
AUDIO_INK> aA]]dicInfo[nKumberO£AudioStream5]; 
DAD^JNFO aDataInfo[nNumibezOfDataStreams]; 
inmjNFO; 



Hie Input/Output Services Definition 

To support record and play needs, a set of high Icvd 
recofd & play services are defined, as described herein- 
above. To support the File for these record and play 
services, the following high-level File VO services are 
defined and described hereinbelow. As will be understood, 
in^lementation of the high level File I/O API is supported 
by a set of "lower level" file I/O Sendee Providers CTSP^s). 
Referring once more to FIG. 6, RFFile 603 may for the 
present exanple be replaced with FileIO___ (not shown). 
When AVDAM 691 requests the creation/opening of a file 
through the Record & Play services, it specifies the required 
file fonnat. Based on the requested file format, FileIO_ 
loads the apprc^iciate Service Provider and carries out all the 
file I/O operations needed with the help of this service 
provider. 

The goals of the File I/O API are: 
Support record/play for sufficiently large number of streams 



This means that there should be support for creating and 
^ adding more number of data types/data streams than the 
ones envisioned now. 

**Opcsr formats for streams-based on. AVI stream formats 
The fonnat for a specific stream is something that should 
not be dictated by the file VO. 
Support for multiple audio, video and **data** streams 

Different products need support for record/play of differ- 
ent number of streams. As an example, some video confer- 
encing products need support fcer at least two video and two 
^ audio streams. Any limit on the number of streams of a given 
type supported will be a lower level file fonnat limitation 
(e.g: the .TAP format limits the # of audio and video streams 
to one each). 

Support for stream-specific information that may be needed 
63 by the stream handler 

It should be possible to add extra infoiroation on a 
per-stream basis; for exainple, annotation on a video stream. 
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Global (FUe scope) stream infoimation File Scope Event Iteck Infonnation 

At the file scope, it should be possible to identify general 

infonnation on streams present in a file. * . ^ . ^ , . . 

File Scope Time'ftack Inf crmation It is useful to have support for oeaUon and usage of 

To facilitate an efficient file-scope/strcam-scope seek ^ "event" maikers in a recorded stream, as described hcrein- 

mechanism, an optional 'TimcT^ack^ stream is utilized. TT above. For this purpose, an EvenflYack(Enrack) stream is 

stream chunks are created by a FilelO.^nableTimelVack utilized. Hie idea is very similar to that of TimcTrack 

(dwTimelntcrvai) call The rate at which TT chunks arc mentioned above; however, the dififeiencc Ues in the fact 

SS^ZTsZ llTs^':^ 10 ^^isnoncaonofanyreg^^^^^ 

quite large in size (discussed below), the TT stream is if there is a fixed pcnodiaty to the Bl^ack, it 

acated with a unique ID and has fixed size chunks for essentially becomes very similar to 'HmeTlrack The idea is 

efficiency purposes. Each chunk in this TT that has the to su{^>ort an event-based search mechanism to all the 

following contents. streams in a FileIO_file. 



rtwctrimeTkacks Etrack is created and updated by a FileIO_ 

DWORD dwTuueStamp: //(relativc/ab9)tune at this point, AddEventTra£k(dwEventlD) calL Thus, the first such call 
TMi/nor. ^™/«u.*nt«-'fu-w ^ «,i??jl?J^Ltu. i- ^ M cfeatcs thc HTrack stream with a unique streamID and 

DWORD dwOffiwtnmel^BclElnfb; //o&et mto TmutlVvck stream ^ ^ 
} TBmjmACK; writing an HTrack stream diunk, defined t>elow as EVENT_ 

TRACIL.CHUNK; subsequent calls result in a ElVack 

Note that each TT chunk has a pointer to a TimeTracklnfb stream chunk to be written into the HTrack stream. 

(TIT) stream offset This TIT stream is just another data 

stream with a unique fourCC code and each chunk of it has 

potentially variable length infonnation. The actual data that 

goes into a TIT diunk is defined as HmelnfoChunk. Each 

TimelnfoChunk has one or more StreamlndexEntry item. 

Each StreamlndexEntry contains information on a stream 30 

that is open at that time. TIT stream consists of one or more 

T^melnfoChunks; the definition of TimelnfoChunk and 

StreamlndexEntry are shown below. 
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struct StrramTnrtnrKntry 
{ 

DWORD dwStteamU; //finnCC 

DWORD dwStreamStEtus; //active, paused . . . 

DWORD dwOfiet; /T'saoirfe IT 

}STREAM_INDE>L.EN1RY; 
struct TimftlnfoChunk 
{ 

DWORD dwNinnlKteiFittiies; //# of atreams "open" dow. . 

STREAMJ3a3GX.^ENT1CY aStreaxnbideacEatriesCdwNuxiibidBzEi^]; 
}TIM«_TRACK-JNPO; 



struct StreamTrvlf.iEntry 

{ 

DWORD dwStresihID; //fouCC for this stKam 

DWORD dwOfeen /rsampJer 

DWORD DwStieamStatas; //active, paused . . . 
}5I1tEAM-JNDGX..ENlltY; /i^ier^txeam-eafty 
struct Eventlnfo 
{ 

DWORD dwNuaolDdexEotries; //# of open streams 

STRBAM^JNDEXJSNIDY aStrramli¥fcTyRntTies(dwNum1hdr¥KTiTTies]; 
}EVENT_INFO; 
struct EwcDttrackS 
{ 

DWORD dwfiveatXD //event ID; uiique 
DWORD dwHoKSttnip; //tnnB at dus point 
EVENXJNFO sBvenibfe; //o^tuies all otber streams* info 
}EVENT_TRACK_CHUNK; 



1 
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File Scope Stream Segment Informatioo 

Each time an open stream is paused/all open streams are 
paused a "segment" information entry is created for the 
streams affected. Such segment information entries are writ- 
ten as file-scope stream segment information, when the file 5 
is dosed. This "segment information stream** will have a 
unique "StreamlD." as will be appreciated by those skilled 
in the art. 
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Returns: 0, if successful 
File Scope Information Management 
Creating/Adding Hmc IVack 

int FileIO_Enablc TimeTrack(IN FILEHANDLE 
hFileHandle, IN DWORD dwUmelnterval) 

dwTimelntcrval — specifies the interval at which each 
TimeTrack entry has to be added; at each such interval^ 



struct SixeamSegDir 
{ 



DWORD 
DWORD 
DWORD 
DWORD 
DWORD 
DWORD 



dwStreamlD; //same as foutCC 

dwScfmeatNuiDber: 

dwStreamState; //active, paused, etc. 



dwSegStaztMaik; 
dwSegEuhoazk; 
dwStreamOffSet. 



//time, event 
//tinie,0vent 
//ofbet into 



}SrrREAXLJSEG_DDL-ENTRY 

struct StreamScgDiil^le 

{ 

DWORD dwNumrHrFjitries 

STREAM_SEG_JDnL.£NlRY aStreainSegDir£ntty[dwNumDir£atries]; 
}STREAM_SEG_DnL_TABLE; 



File I/O AH 

Various functions that are required to implement the file 
VO API arc grouped together as follows: 
Libsary scope operations — IniHah'yatc/Shutdown 
File scope functions — opening/closing operations, etc. 
File scope informatioB — read/wiite c^>erations 
Stream management— opening/closing/creation of 
streams 

Stream input/output— writcA-cad operations 
Stream-sepcific information— write^ead stream specific 
infonxiation 

Library Scope Initialization/Shutdown Operations 
Initialization 

int FilcIO_Init(void) 

Inilalizes the FilcIO_subsystcm. 

Return value — 0, if successful 
Shutdown 

int FilcIO_JShutDown(void) 

Performs the dcan-up and shutdown of the FileIO„ 
subsystem. 

Return value — 0, if successful 
File scope operations 
Opening a file 

LONG FilcIO_Open (OUT FILEHANDLE 
FAR*lphFilcHandle, IN LPCSTR IpPath, IN UINT 
uFonnat IN UINT uMode) 
Arguments: 

^hFlleHandie— a pointer to a FILEHANDLE; the loca- 
tion pointed to by IphFileHandle used to place a 
non-zero FILEHANDLE value if the operation is suc- 
cessful 

IpPath — points to a null terminated ASCII string that 
contains the path, including the file name, of the file to 
be opened 

uFormat — specifies the "low levd** format (.TAP, JVVI, 

.PCS) that this file to be opened uses. 
uMode-^pedfics the mode with which the file is to be 

opened 
Qosing a file 

LONG FdeIO_aose(IN FILEHANDLE hFileHandle) 
Arguments: 

hFQeHandlc — specifies a valid FILEHANDLE, returned 
by a previous call to FileIO_Open 



25 a Ttmetrack entry with appropriate stream information 
(offset, active^used status etc.) for each open stream 
is added. 
Creating^Adding Event Track 

int FileIO_AddEvcntTrack(IN FILEHANDLE 

hFileHandle, IN DWORD dwEventID) 

dwEventID — unique Event marker. Creates/adds an 
Eventl^ack entry, with the current stream inf onnation 
(offsets, active/paused status etc. ) for each stream that 
is open and cuirmt time. 

General file information 
In addition to die above-specified file-scope data, a global 

file-scope inftamatiott chunk (defined below in FIL£_JNFO 

structure) is created and added with a unique StreamID, 

when the file is dosed. 

int FfleIO_WrlteFileInfo(IN FILEHANDLE hFileHandle, 
40 JS LP_JFlLE_JNFO ^Filcilnf o) 

This infonnation is intended to be used during playback, 
to assist the Record&Playback services and its dients with 
infonnation on the recorded file. 

Aprototype of the FILELJNFO structure is shown below. 

45 



tio Reading FUelnfo 

int FileIO_J^eadFileInfo(]N FILEHANDLE hFileHandle, 
IN LP J1LE_JNF0 ^elnfo) 

This function reads FILE.JNFO data present in a file 
opened for READ. 
65 hHleHandle— valid file handle 

IpFHelnfo— a pointer to FILE...JNFO structure, to read 
the data into. 



Unicff'ifeliifoT^g 

{ 
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UIOT 


iiFormat; 


int 


nNiiDbecO£Streaim; 


BOOL 


blsneTrackAeaent; 


DWORD 


dwNumXIme'naclEntries; 


BOOL 


bEventlbackPtesent; 


DWORD 


dwNinnEvenirnBckEntrks; 


int 


nNumbecOfVkfeoStteamB; 


int 


uNumbocOCAudiDStreams; 


int 


nNumboOfDataStreams: 



VIIffiO_JNFO a>nckobfiD(iiNuinbefOfVideo5treaiiBj; 
AUDIO_JNFO aAudiflInf6[nNumbeiOfAudic>Streaiits]; 
DATAJNFO aDaUliifo{nNumbeiO £DataStrean» ]; 
}FILEJNFO; 
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Creating and wiiting addidonal file scope data: Balance each call to FildO^OpenStream with a call to 

IX>NG FilcIO_WriteData(IN FILEHANDLB hPileHandle, FilcIO_Streamaose using the stream handle returned 
IN DWORD dwStrcamlD, IN LPVOID lpData» IN LONG Qosing a Stream 

<*I>ata) LONG FileIO_StreamClose{IN FileIO_STREAM 

Writes some additional data to the file. 5 pFileIO_JStream) 

Returns zoo if successful; otherwise, it returns an error Qoses an open stream specified by pFildO _Stream 

Returns the cuirent reference count of the stream. This 
hFilcHandle— Specifics a handle to an open FileIO_filc. value should only be used for debugging puipbses. 
^^l^f^^P^"^^ four-character code identifying pFileIO_Stream^pecifies a handle to an open stream, 
me data. jq i^g^jjjjg ^ stream header 

^ ^ LONG FilelO_ReadStteamInfo(FileIO_STREAM 
cbData—Speafies the size of IheinemQiy block refeired pFileIO_Strcam, LPFileI0_STREAMINFO FAR*1 

nSSLS'Llswith.ret^nvalueofFnelO.ERK^ ' oSifSS!^^SSL 
RE.DONLYunless««fl,cwascpcncdwid.wrl.ep««us-, J^^r^Ssc. it retun. an «ror 

Reading file scope additional data 

LONG FileIO_ReadData(IN FILEHANDLE hFileHandle, pHleIO_Strcam-^pecifics a handle to an open stream. 

IN DWORD dwStreamID, OUT LPVOID IpData, OUT pFilelO.Strcamlnfo-^pedfies a painter to a structure 

LONG FAR*IpcbData) used to return stream information. 

Reads optional data from the file. ^ ISiz^-nSpecifies the size of the structure used for 

Returns zero if successful; otherwise, it returns an error LPFildO STOEAMINFO 

code.mr^ valueFik^^ Stream Input/^tput Wiite/tcad Operations 

dato with the requested four-character code (dwStreamID) Reading Data frwn a Stream 

^^M^^^f c u A^ * 17M rr. *;i M LONG FileIO_StreamRcad(IN FileIO_STREAM 

f t!?'"*^^^^^ ^"^1^ ""^^^f^:^"" hFilelO^tream, IN LONG iStart, IN LONG ISamples, 

dwStr^mlD-Speofies a four^haractcr code idcntifymg qUT LPVOID ^)Buflfcr, IN LONG cbBufiFcr, OUT LONG 

tile data, ^ ^ , ^ FAR'PlBytes, OUT LONG FAR'plSan^des) 

IpData— Specifics a pointer to a buffer used to return the Reads data from a stream. 

data read. Returns zero if successful; otherwise, it returns an error 

IpcbData— Specifies a pointer to a location indicating the * code. Use i^ytes and plSamples to find out how much was 

size of the memory block referred to by IpData. If the actually read. 

read is successful, the value is changed to indicate the Possible errors include: 

i^^.T^'^''^'^.^^ ' /I • / «Pc^ FaeIO_ERR_3UFFEKrOOSMALL, FileIO_ERR_ 

toamManagcinaitQpcmng/closmg/acaU MEMORY, FilcIO_£RRJILEREAD 
Creatmg a stream (for WRITE) " Anrumcnts- 

LONG FileIO_CreateStream(IN FILEHANDLE * rr. cL c ^ u * 

hFileHandle, OUT LP_FileIO_STREAM hFileIO_Strcam-Specifies a handle to an open stream. 

FAR»lppFileIO_Strcam, IN FacIO_STREAMINFO IStart— Specifies the starting "sanq)lc" to read. For audio 

FAR*lpiFQeIO_JStreamInfo) san^les correspond to a block of audio data; for 

Creates a new stream in an existing file, and returns a ^ ^"^^^ ^^^^ * 

streanarinterface pointer for it ISamples — Specifies the number of "sainples** to read. 

Returns zero if successful; otherwise, it returns an error IpBuffcr — Specifies a pointer to a buffer used to return the 

code. data. 

hFileHandle — Specifies a handle to an open FileIO_file. cbBuffier-^pecifies the size of buffer pointed to by 

^FileIO_J5tceam— Specifies a pointer used to return die IpBuffer. 

new stream-interface pointer. IplBytcs— Specifies a pointer to the location used to 

lpFileIO_StreamInfo— Specifies a pointer to a structure return the number of bytes of data written into the 

containing information about the new stream. This buffer for IpBuffer. plBytes can be NULL, 
structure contains the type of the new stream and its ^ IplSamples — Specifies a pointer to the location used to 

sample rate. return the number of samples written into tiie buffer for 

This function fails with a return value of FilelO JERR_ IpBuffer. plSamples can be NULL. 

READONLY unless the file was opened with write permis- If IpBufferis NULL, this function docs not read any data; 

sion. it returns information about the size of data that would be 
Opening a stream (for READ) ^3 read. 

LONG FileIO_OpenStream(IN FILEHANDLE Writing Data into an Existing Stream 

hFileHandle, OUT PFUeIO_STREAM FAR*ppFilcIO_ LONG FilcIO_StrcamWrite(IN FilcIO_STREAM 

Stream, IN DWORD dwStreamlD) hFacIO_JStream, IN LONG IStait, IN LONG lSaiiq)lcs, IN 

Returns zero if successful; otherwise, it returns an error LPVOID IpBuffer, IN LONG cbBuffer, IN DWORD 

code. Possible error codes include: (no stream in the file dwFlags, OUT LONG FAR* lpISan5>Written, OUT LONG 

corresponding to the values passed in for dwStzeamlD.) FAR*lplBytes Written) 

hFileHandle — Specifies a handle to an open FilelO^JUe. Writes data to a stream. 

ppFilelO.Stream— Specifies a pointer to the return loca- Returns zero if successful; otherwise, it returns an error 

tion for the new stream-interface pointer. code. 
dwStreamlD — Specifies an ID (four-character code) indi- 65 hFileIO_J5tteam — Specifies a handle to an open stream. 

eating die type of stream to be opened. Zero is a IStart— Specifies die starting san^>le to write. 

reserved stream id. ISamples — Specifies the number of samples to write. 
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IpBuffer — Specifies a pointer to buffer containing the data 
to write. 

cbBuffer — Specifies the size of buffer used by IpBuffcr. 
dwFlags — Specifies any flags associated with this data. 

The following flag is defined for video streams: 

FileIO_P_KEYFRAME 
IplSampWrittcD — Specifies a pointer to a location used to 

return the number of sanities written. This can be set to 

NULL. 

IplBytes Written — Specifies a pointer to a location used to 
return the number of bytes written. This can be set to 
NULL. 

Stream-specific Information: Write/read Stream Specific 
Information 

Reading optional stream header information 
LONG FilcI0_5treamReadData(IN FacIO_STREAM 
hFileIO_Stream, IN DWORD dwStrcamId, OUT LPVOID 
IpData, OUT LONG FAR*lpcbData) 

Reads optional header data from a stream. 

Returns zero if successful; otherwise, it returns an error 
code. The return value FileIO_JERtL_NODArA indicates 
the system could not find any data with the specified chunk 
ID. 

hFileIO_Stream — Specifies a handle to an open stream. 
dwStrtamlD — Specifies a four-character code identifying 
the data. 

IpData — Specifies a pointer to be used to return the data 
read. 

IpcbData — Points to a location which specifies the buffer 
size used for IpData. If the read is successful, FileIO_ 
changes this value to indicate the amount of data 
written into the buffer for IpData. 
Writing Additional Stream Data 

LONG FiicIO__Stream WriteData(IN FileIO_STREAM 
hFileIO_Stream, IN DWORD dwStrcamId, IN LPVOID 
IpData, IN LONG cbDaU) 

Writes optional data to the stream. 

Returns zero if successful; otherwise, it returns an error 
code. 

hFileIO_Stream — Specifies a handle to an open streaoL 
dwStreamld — Specifies a four-character code identifying 
the data. 

IpData — Specifies a pointer to a buffer containing the data 
to write. 

cbData — Indicates the number of bytes of data to be 
copied from IpData into the stream. 

This call fails with a return value of FileIO__ERR_ 
READONLY unless the file was opened with write permis- 
sion. 

As will be understood, the present invention can be 
embodied in the form of computer-in^lemented processes 
and apparatuses for practiciiig those processes. The present 
invention can also be embodied in the form of computer 
program code embodied in tangible media, such as floppy 
diskettes, CD-ROMs, hard drives, or any other coix^uter- 
readable storage medium* wherein, when the computer 
program code is loaded into and executed by a computer, the 
oonQ>uter t>ecomes an apparatus for practicing the invention. 
The present invention can also be embodied in the form 
ooiX4>uter program code, for example, whether stored in a 
storage medium, loaded into and/or executed by a computer, 
or transmitted over some transmission medium, sudi as over 
electrical wiring or cabling, through fiber optics, or via 
electromagnetic radiation, wherein, when the computer pro- 
gram code is loaded into and executed by a conq)uter, the 
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computer becomes an apparatus for practicing file invention. 
When inq)lemented on a general-purpose microprocessor, 
the computer program code segments configure file micro- 
processor to CTeate specific logic circuits. 

S It will be understood fiiat various changes in the details, 
materials, and arrangements of the parts which have been 
described and illustrated above in order to explain the nature 
of this invention may be made by (hose skilled in the an 
without departing from the principle and scope of the 

10 invention as recited in the following claims. 
What is claimed is: 

1. A method for event tagging of at least one audio, video, 
or data (AVD) stream, eadi ^VD stream oonqiri&ing a 
plurality of AVD stream chunks, the method oon^sing the 

15 steps of: 

(a) storing the at least one AVD stream; and 

(b) storing an event tag in response to a signal indicating 
the beginning of an event, the event tag having an event 
stream information data (ESID) structure comprising 
an ESID entry for each AVD stream of the at least one 
AVD stream, each ESID entry comprising an offset for 
locating a conesponding AVD stream chunk of each 
active AVD stream of the at least one AVD stream. 

2. The method of claim 1, wherein the signal is generated 
^ by a user. 

X The method of claim 1, wherein the signal is generated 
automatically by the conqxiter in accordance widi a prese- 
lected event type. 

4. The method of claim 1, the event tag ftirther comprising 
^ an event identifier for identifying the event and a time stamp 

for indicating the time of the corresponding AVD stream 
chunk fox each of the at least one AVD stream ooiresponding 
to the beginning of the event 

5. The method of claim 4, wherein the time stamp 
indicates the absolute time of the corresponding AVD stream 
chunk for each of the at least one AVD stream corresponding 
to the beginning of the event 

6. The method of daim 1, wherein: 

the ESID structure further comprises a number of entries 
field for indicating the number of ESID entries in the 
ESID structure; and 

each ESID entry further con^ses a stream identifier field 
for identifying fiie coiresponding AVD stream and a 
stream status field for indicating the current status of 
the conesponding AVD stream. 

7. The mefiiod of claim 6, the event tag fiirther comprising 
an event identifier for identifying the event and a time stamp 
for indicating the time of the oorrcsponding ^yD stream 

^ chunk for each of the at least one AVD stream corresponding 
to the beginning of the event 

8. The method of claim 7, wherein the event identifier is 
unique and is mapped to a symbolic name that describes the 
event 

9. The method of claim 7, wherein the status field of each 
Sn>S chunk entry indicates whether the corresponding AVD 
stream is active paused, active unpaused, or inactive. 

10. The method of claim 7, wherein: 

the at least one AVD stream oonpsrises at least two AVD 
^ streams of a data conference between at least two users; 
each AVD stream of the at least two /^/D streams has a 

unique conference identifier, and 
the identifier field of each SIDS chunk entry identifies the 
unique conference identifier for the corresponding 
65 AVD stream. 

11. The method of claim 7, wherein: 

the offset of each ESID entry is a 32-bit pointen 
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the stream identifier field of each ESID entry is a 32-but 

stream identifier field; 
the stream status field of each ESID entry is a 32-but 

stream status field; 
the event identifier is a 32-bit event identifier field; 
the time stamp is a 32-bit time stamp field; and 
the number of entries field is a 32-bit number of entries 

field. 

12. The method of claim 1, further conqvising the step of: 
(c) stating a sequence of event tags for a plurality of 

ccxresponding events in an event track. 

13. The method of claim 1, wherein each AVD chunk of 
an AVD stream of the at least two AVD streams is a chunk 
of data in accordance with whether the AVD stream is an 
audio, video, or data stream. 

14. The method of daim 13, wherein each AVD chunk of 
an AVD stream of tiie at least two AVD stceanis is a video 
frame if the each AVD chunk is a video stream and an audio 
sample if the each AVD chunk is an audio streauL 

15. An apparatus for event tagging of at least one audio, 
video, or data (AVD) stream, each stream conqnising 
a plurality of AVD stream chunks, the apparatus con^irising: 

(a) means for staring the at least one AVD stream; and 

(b) means for storing an event tag in response to a signal 
indicating the beginning of an event, die event tag 
having an event stream information data (ESID) struc- 
ture con^vising an ESID entry for each AVD sdream of 
the at least one AVD stream, each ESID entry com- 
prising an offset for locating a corresponding AVD 
stream chunk of each active AVD stream of the at least 
one AVD stream. 

16. The apparatus of claim 15, wherein the signal is 
generated by a user. 

17. The apparatus of daim 15, whcrdn the signal is 
generated automatically by the computa In accordance with 
a preselected event type. 

18. The apparatus of claim 15, the event tag fiirther 
ooin{nising an event identifier for identifying the event and 
a time stan^ for indicating the time of the corresponding 
AVD stream chunk for each of the at least one AVD stream 
corresponding to the beginning of the event 

19. The apparatus of daim 18, wherein the time stan^ 
indicates the absolute time of the corresponding AVD stream 
diunk for each of the at least one AVD stream ccxrresponding 
to the beginning of die event 

20. The apparatus of daim 15, wherein: 

the ESID structure further comprises a number of entries 
field for indicating the number of ESID entries in the 
ESID structure; and 

each ESID entry further comprises a stream identifier field 
for identifying the oonesponding AVD stream and a 
stream status field for indicating the current status of 
the oooresponding AVD stream. 

21. The appsTBtus of daim 2t, the event tag further 
comprising an event identifier for identifying the event and 
a time stamp for indicating the time of the cxuresponding 
AVD stream chunk for eadi of the at least one AVD stream 
cairespoitding to die b^inning of the event 

22. The q>paratus of daim 21, wherein the event identifier 
is unique and is mapped to a symbolic name diat describes 
the event. 

23. The ^iparatus of claim 21, wherein the status fidd of 
each SIDS chunk entry indicates whether the corresponding 
AVD stream is active paused, active unpaused, or inactive. 

24. Ihe appanxas of daim 21, wherein: 

the at least one AVD stream comprises at least two AVD 
streams of a data conference between.at least two users; 
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each AVD stream of the at least two AVD streams has a 

unique conference identifier; and 
the identifier fidd of each SIDS chunk entry identifies the 
unique conference identifier for the ccnesponding 
AVD strcanL 

25. The apparatus of claim 21, wherein: 
die offset of each ESID entry is a 32-bit pointcn 
die stream identifier fidd of each ESID entry is a 32-but 

stream identifier fidd; 

die stream status fidd of each ESID entry is a 32-bttt 

stream status field; 
die event identifier is a 32-bit event identifier field; 
the time stamp is a 32-bit time sUaxxp fidd; and 
die number of entries field is a 32-bit number of entries 
field. 

26. The apparatus of daim 15, further comprising: 

(c) means for storing a sequence of event tags f<x a 
plurality of corresponding events in an event track. 

27. The appfastas of claim 15, wherein each AVD chunk 
of an AVD stream of the at least two AVD streams is a chunk 
of data in accordance with whedier die AVD stream is an 
audio, video, or data stream. 

28. The ^xparatus of daim 27, wherein each AVD chunk 
of an AVD stream of the at least two AVD streams is a video 
frame if the each AVD chunk is a video stream and an audio 
sample if the each AVD chunk is an audio stream. 

29. A con^Hiter-readable medium having stored thereon a 
plurality of instructions for event tagging of at least one 
audio, video, or data (AVD) stream, each, stream 
con^srising a plurality of AVD stream chunks, wherein the 
plurality of instractions, when executed by a processor of a 
computer, cause the processor to perform the stq)s of: 

(a) storing the at least one AVD stream; and 

(b) staring an event tag in response to a signal indicating 
the beginning of an event, die event tag having an event 
stream infocmatioa data (ESID) structure comprising 
an ESID entry for each AVD stream of die at least one 
/^D stream, each ESID entry comprising an offset for 
locating a cocresponding AVD stream chunk of each 
active AVD stream of the at least one AVD streant 

3t. The coiqwter-ieadable medium of claim 29, whaein 
die signal is generated by a user. 

3L The counter-readable medium of daim 29, wherein 
the signal is generated automatically by the computer in 
accordance widi a presdcctcd event type. 

32. The computer-readable medium of daim 29, the event 
tag further comprising an event identifier for identifying die 
event and a time stan^) for indicating the dme of the 
corresponding AVD stream chunk f(» each of the at least one 
AVD stream coiresponding to die beginning of the event 

33. The computer-readable medium of daim 32, wherein 
the time stamp indicates the absolute time of the correspond- 
ing AVD stream diunk for each of the at least one AVD 
stream oosresponding to the beginning of the event. 

34. The con^Miter-readable medium of claim 29, wherein: 
the ESID structure further comprises a number of entries 

field for indicating the number of ESID entries in the 
ESID structure; and 
eadi ESID entry further comprises a streamidentifier field 
for identifying the coiresponding AVD stream and a 
stream status fidd for indicating the current status of 
the coiresponding AVD streant 

35. The con9)uter-readable medium of claim 34, the event 
tag further comprising an event identifier for identifying die 
event and a time stamp for indicating the time of the 
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ooiresponding AVD stream chunk for each of the at least one 
AVD stream conresponding to the beginning of the event. 

36. The conq}uter-readable medium of daim 35, wherein 
the event identifier is unique and is mapped to a symbolic 
name that describes the event. 

.37. The computer-readable medium of claim 35, wherein 
the status field o[ each SIDS chunk entry indicates whether 
the corresponding AVD stream is active paused, active 
unpaused, cr inactive. 

38. The computer-readable medium of claim 35, wherein: 
tiie at least one AVD stream comprises at least two AVD 

streams of a data conference between at least two users; 
each AVD stream of the at least two AVD streams has a 

unique conference identifier, and 
the identifier field of eadi SIDS chunk entry indentifies 

the unique conference identifier for the corresponding 

AVD strcancL 

39. The computer-readable medium of daim 35, wherein: 
the offset of each ESID entry is a 32-bit pointer, 

the stream identifier fidd of each ESID entry is a 32-but 

stream identifier fidd; 
the stream status fidd of each ESID entry is a 32-but 

stream status fidd; 
the event identifier is a 32-bit event identifier field; 
the time stamp is a 32-bit time stamp field; and 
the nuniber of entries fidd is a 32-bit number of entries 

field. 

40. The computer-^^dable medium of daim 29, v^erein 
the plurality of instructions cause the processor to perform 
the step of: 

(c) storing a sequence oi event tags for a plurality of 
corresponding events in an event track. 

41. The computer-readable medium of daim 29. wherein 
each AVD chunk of an AVD stream of the at least two AVD 
streams is a chunk of data in accordance with whether tiie 
AVD stream is an audio, video, or data stream. 

42. The computer-readable ixkedium of daun 41, wherein 
each AVD chunk of an AVD stream of the at least two AVD 
streams is a video frame if the each AVD chunk is a video 
stream and an audio sample if the each AVD chunk is an 
audio stream. 

43. A conoputer system, comprising: 
a processor, and 

a memory coupled to the processor, wherein tibe memory 
stores a plurality of instructions for event tagging of at 
least one audio, video, or data (AVD) stream, each AVD 
stream comprising a plurality of AVD stream chunks, 
wherein: 

(a) the at least one AVD stream is stored; and 

(b) an event tag is stored in response to a signal 
indicating the beginning of an event, the event tag 
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having an event stream infonsation data (ESID) 
stnicture comprising an ESID entry for each AVD 
stream of the at least one AVD stream, each ESID 
entry comprising an offset for locating a cocrespond- 
3 ing AVD stream chunk of each active AVD stream of 

the at least one AVD streaoL 

44. The con^ter system of daim 43, die event tag further 
comprising an event identifier for identifying the event and 
a time stamp for indicating the time of the corresponding 

to AVD stream chunk for each of tiie at least one AVD stream 
corresponding to the beginning of the event 

45. The coaq)Utcr system of daim 43, wherein: 

the ESID structure further comprises a number of entries 
field for indicating the number of ESID entries in the 
ESID stmcture; and 
each ESID entry further con^irises a stream identifier field 
for identifying the conesponding AVD stream and a 
stream status fidd for indicating the current status of 
2Q the corresponding AVD streanx 

46. Hie computer system of claim 45, the event tag further 
comprising an event identifier for identifying the event and 
a time stamp for indicating the time of the corresponding 
AVD stream diunk for each of the at least one ^D stream 

^ coresponding to the beginning <^ the event 

47. The computer system of daim 46, wherein the event 
identifier is unique and is mapped to a symbolic name that 
describes the event 

48. The computer system of daim 46, wherein the stams 
^ fidd of each SIDS chunk entry indicates whether the cor- 
responding pyu stream is active paused, active uiq>aused, 
or inactive. 

49. The conqjuter system of daim 46, wherein: 

the at least one AVD stream ooinprises at least two AVD 
35 Streams of a data conference between at least two users; 
each AVD stream of the at least two AVD streams has a 

unique conference identifier; and 
the identifier fidd of each SIDS chunk entry identifies the 
unique conference identifier for the corresponding 
AVD stream. 

50. The computer system of daim 46, wherein: 
the offset of cadi ESID entry is a 32-bit pointcn 
the stream identifier fidd of each ESID entry is a 32-but 

stream identifier fidd; 
the stream status fidd of each ESID entry is a 32-but 

stream status field; 
the event identifier is a 32-bit event identifier field; 
the tiime stamp is a 32-bit time stamp fidd; and 
^ the number of entries field is a 32-bit number of entries 
fidd. 
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